Papers with exact pipeline of model distillation
Efficient Active Learning with Adapters (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that distilled versions of pretrained models are not always available. |
| Approach: | They propose to use distilled versions of successor models as acquisition models to reduce the training cost of the model. |
| Outcome: | The proposed approach reduces the training cost of the model and does not cause the acquisition-successor mismatch (ASM) problem. |